Method and apparatus for adjusting a model of an anatomical structure

ABSTRACT

There is provided a method and apparatus for adjusting a model of an anatomical structure in a sequence of images of the anatomical structure. The model is placed with respect to the anatomical structure in the sequence of images. A user input is received to adjust the model in a selected image of the sequence, and based on the user input, a part of the model that lies in the selected image and a previously unadjusted part of the model that lies in one or more other images of the sequence is adjusted, whilst fixing in place a previously adjusted part of the model that lies in other images of the sequence.

TECHNICAL FIELD OF THE INVENTION

This disclosure relates to the field of image processing and, inparticular, to a method and apparatus for adjusting a model of ananatomical structure in a sequence of images of the anatomicalstructure.

BACKGROUND OF THE INVENTION

Medical image processing often uses segmentation to determine thelocation of features of interest in a medical image and to generate apatient-specific model of the anatomical structure in the image. Thispatient-specific model can then be used to make measurements of theanatomical structure (for example, to measure the left ventricularvolume over the heart cycle), for diagnosis, treatment planning (forexample, in aortic valve implant fitting), or for the prediction ofdisease (for example, biophysical models). In order to extract the shapeof an anatomical structure from medical images, different methods ofsegmentation can be used, such as, for example, a model-basedsegmentation framework in which an adjustable model of the anatomicalstructure is placed in the image to segment the anatomical structure.

An example of such a model-based segmentation framework is provided inan article by Ecabert et al., entitled “Segmentation of the heart andgreat vessels in CT images using a model-based adaptation framework,Medical Image Analysis”, Volume 15, Issue 6, December 2011, Pages863-876, which describes detailed local correction and manual editing ofa multi-color mask overlain on an image of a heart.

However, in these existing techniques, if the segmentation of ananatomical structure in an image is inaccurate (for example, if thesegmentation results in a model that does not accurately reflect theshape of the anatomical structure in the image), various correctionmechanisms are available whereby a physician can manually adjust themodel to correct the segmentation result. For example, a physician maymanually adjust the model by dragging points of the model towards thecorrect features of the anatomical structure in the image. For thepurposes of efficiency, existing correction methods typically modify themodel in multiple images simultaneously as this reduces the manual workinvolved and can keep the segmentation consistent across the images. Forexample, a modification may be propagated through multiple images intime-series of two-dimensional images, or through multiple images intime-series of three-dimensional images and/or in all three-dimensionsin the case of three-dimensional images.

As a result, not only parts of the model belonging to a currentlydisplayed image are modified, but also parts of the model belonging toneighboring images in the sequence are modified. This means that,following an adjustment to the model in an image, the physician mustreview neighboring images to ensure that any changes that werepropagated through to the neighboring images are reasonable. In someinstances, for example, the physician will need to review images thathave already been previously inspected to determine whether anymodifications that have propagated to those previously inspected imageshave resulted in errors that require the model to be re-adjusted inthose images. In order to ensure that no errors result from amodification to the model in one image, the physician would need tocheck the model in all other images.

In the case of adjusting a model in a four-dimensional image comprisinga time-sequence of three-dimensional images, the propagation ofadjustments to the model across the time-sequence of three-dimensionalimages can occur in both space (for example, to images at differentpoints in space, such as neighboring images) and time (for example, toimages at different points in time). In such examples, it becomes evenmore complicated and time consuming to review adjustments to the modelas the adjustments may have been propagated in both space and time.

Thus, the process for adjusting a model of an anatomical structurebecomes inefficient once again and is also complex. There is thus a needfor an improved method and apparatus for adjusting a model of ananatomical structure in a sequence of images of the anatomicalstructure.

SUMMARY OF THE INVENTION

As noted above, a limitation with existing techniques for adjusting amodel of an anatomical structure in a sequence of images of theanatomical structure is that the techniques result in errors that canpropagate through multiple images in the sequence and the techniques arealso inefficient and overly complex. It would thus be valuable to havean improved method for adjusting a model of an anatomical structure,which overcomes the problems described above.

Therefore, according to a first aspect of the invention, there isprovided a method for adjusting a model of an anatomical structure in asequence of images of the anatomical structure, the method comprising:placing the model with respect to the anatomical structure in thesequence of images; receiving a user input to adjust the model in aselected image of the sequence; and adjusting, based on the user input,a part of the model that lies in the selected image and a previouslyunadjusted part of the model that lies in one or more other images ofthe sequence, whilst fixing in place a previously adjusted part of themodel that lies in other images of the sequence.

In some embodiments, the previously unadjusted part of the model lies inone or more images that are subsequent to the selected image in thesequence and the previously adjusted part of the model lies in imagesthat precede the selected image in the sequence.

In some embodiments, the method further comprises fixing in place theadjusted part of the model that lies in the selected image forsubsequent adjustments.

In some embodiments, adjusting a previously unadjusted part of the modelthat lies in the one or more other images of the sequence comprisesusing an interpolation to adjust the previously unadjusted part of themodel that lies in the one or more other images of the sequence, basedon the adjustment to the part of the model that lies in the selectedimage.

In some embodiments, the method further comprises receiving at least onefurther user input to adjust the model in at least one further image inthe sequence; and adjusting, based on the at least one further userinput, a part of the model that lies in the at least one further image,whilst fixing in place a previously adjusted part of the model that liesin other images of the sequence.

In some embodiments, the one or more other images of the sequence inwhich a previously unadjusted part of the model is adjusted are selectedbased on a property of the anatomical structure.

In some embodiments, the one or more other images of the sequence inwhich a previously unadjusted part of the model is adjusted lie within apredetermined radius of the selected image.

In some embodiments, the sequence of images comprises a time sequence oftwo-dimensional images, wherein each two-dimensional image in thesequence shows the anatomical structure at a subsequent point in timefrom the preceding two-dimensional image in the sequence.

In some embodiments, the sequence of images comprises a sequence oftwo-dimensional slices through a three-dimensional image, wherein eachslice of the three-dimensional image in the sequence shows theanatomical structure at a subsequent two-dimensional spatial plane fromthe preceding slice of the three-dimensional image in the sequence.

In some embodiments, the sequence of images comprises a time sequence ofthree-dimensional images, wherein each three-dimensional image in thesequence shows the anatomical structure at a subsequent point in timefrom the preceding three-dimensional image in the sequence.

In some embodiments, the model of the anatomical structure comprises amesh comprising a plurality of segments and the method further comprisesdetermining which of the plurality of segments lie in the selected imagein the sequence; and adjusting, based on the user input, one or more ofthe segments that are determined to lie in the selected image in thesequence.

In some embodiments, the one or more adjusted segments lying in theselected image comprise at least one segment of the mesh to which thereceived user input relates and one or more adjacent segments.

According to a second aspect of the invention, there is provided acomputer program product comprising a computer readable medium, thecomputer readable medium having computer readable code embodied therein,the computer readable code being configured such that, on execution by asuitable computer or processor, the computer or processor is caused toperform the method described above.

According to a third aspect of the invention, there is an apparatus foradjusting a model of an anatomical structure in a sequence of images ofthe anatomical structure, the apparatus comprising: a processorconfigured to: place the model with respect to the anatomical structurein the sequence of images; receive a user input to adjust the model in aselected image of the sequence; and adjust, based on the user input, apart of the model that lies in the selected image and a previouslyunadjusted part of the model that lies in one or more other images ofthe sequence, whilst fixing in place a previously adjusted part of themodel that lies in other images of the sequence.

In some embodiments, the processor is configured to control a userinterface to render the adjusted model of the anatomical structure;and/or control a memory to store the adjusted model of the anatomicalstructure.

According to the aspects and embodiments described above, thelimitations of existing techniques are addressed. In particular,according to the above-described aspects and embodiments, a model of ananatomical structure in a sequence of images of the anatomical structurecan be adjusted in a simple and efficient manner, while errorspropagating through multiple images are avoided to produce more reliableadjusted models of anatomical structures.

There is thus provided an improved method of adjusting a model of ananatomical structure in a sequence of images of the anatomicalstructure, which overcomes the existing problems.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the embodiments, and to show more clearlyhow they may be carried into effect, reference will now be made, by wayof example only, to the accompanying drawings, in which:

FIG. 1 is a block diagram of an apparatus for adjusting a model of ananatomical structure according to an embodiment;

FIG. 2 illustrates a method for adjusting a model of an anatomicalstructure according to an embodiment; and

FIG. 3 illustrates an example embodiment of a method of adjusting amodel of an anatomical structure in use.

DETAILED DESCRIPTION OF THE EMBODIMENTS

FIG. 1 shows a block diagram of an apparatus 100 according to anembodiment that can be used for adjusting a model of an anatomicalstructure in a sequence of images of the anatomical structure.

With reference to FIG. 1, the apparatus 100 comprises a processor 102that controls the operation of the apparatus 100 and that can implementthe method described herein. The processor 102 can comprise one or moreprocessors, processing units, multi-core processors or modules that areconfigured or programmed to control the apparatus 100 in the mannerdescribed herein. In particular implementations, the processor 102 cancomprise a plurality of software and/or hardware modules that are eachconfigured to perform, or are for performing, individual or multiplesteps of the method according to the embodiments described herein.

Briefly, the processor 102 is configured to place the model of theanatomical structure with respect to the anatomical structure in thesequence of images, receive a user input to adjust (or modify) the modelin a selected image of the sequence, and adjust (or modify), based onthe user input, a part (or portion) of the model that lies in theselected image and a previously unadjusted part (or portion) of themodel that lies in one or more other images of the sequence, whilstfixing in place a previously adjusted part (or portion) of the modelthat lies in other images of the sequence.

In this way, any parts of the model that have been previously adjustedby the user are fixed (or frozen) in place. This eliminates the need forthe user to have to repeatedly check and re-adjust sections of the modelthat have already been adjusted and approved by the user.

In some embodiments, the apparatus 100 may also comprise at least oneuser interface 104. Alternatively or in addition, at least one userinterface 104 may be external to (i.e. separate to or remote from) theapparatus 100. For example, at least one user interface 104 may be partof another device.

A user interface 104 may be for use in providing a user of the apparatus100 (for example, a healthcare practitioner, a healthcare specialist, acare giver, a subject, or any other user) with information resultingfrom the method according to embodiments herein. The processor 102 maybe configured to control one or more user interfaces 104 to provideinformation resulting from the method according to embodiments herein.For example, the processor 102 may be configured to control one or moreuser interfaces 104 to render (or output or display) one or more of theimages in the sequence of images and/or the adjusted model of theanatomical structure. Alternatively or in addition, a user interface 104may be configured to receive a user input. In other words, a userinterface 104 may allow a user of the apparatus 100 to manually enterinstructions, data, or information. For example, in some embodiments,the user interface 104 may allow the user of the apparatus 100 toprovide a user input to adjust the model of the anatomical structure inthe manner described herein. The processor 102 may be configured toacquire the user input from one or more user interfaces 104.

A user interface 104 may be any user interface that enables rendering(or output or display) of information, data or signals to a user of theapparatus 100. Alternatively or in addition, a user interface 104 may beany user interface that enables a user of the apparatus 100 to provide auser input, interact with and/or control the apparatus 100. For example,the user interface 104 may comprise one or more switches, one or morebuttons, a keypad, a keyboard, a touch screen or an application (forexample, on a tablet or smartphone), a display screen, a graphical userinterface (GUI) or other visual rendering component, one or morespeakers, one or more microphones or any other audio component, one ormore lights, a component for providing tactile feedback (e.g. avibration function), or any other user interface, or combination of userinterfaces.

In some embodiments, the apparatus 100 may also comprise a memory 106configured to store program code that can be executed by the processor102 to perform the method described herein. Alternatively or inaddition, one or more memories 106 may be external to (i.e. separate toor remote from) the apparatus 100. For example, one or more memories 106may be part of another device. A memory 106 can be used to store modelsof anatomical structures, images of anatomical structures, information,data, signals and measurements acquired or made by the processor 102 ofthe apparatus 100 or from any interfaces, memories or devices that areexternal to the apparatus 100. For example, a memory 106 may be used tostore one or more of the images in the sequence of images and/or theadjusted model of the anatomical structure. The processor 102 may beconfigured to control a memory 106 to store the adjusted model and/orthe images from the sequence of images.

In some embodiments, the apparatus 100 may also comprise acommunications interface (or circuitry) 108 for enabling the apparatus100 to communicate with any interfaces, memories and devices that areinternal or external to the apparatus 100. The communications interface108 may communicate with any interfaces, memories and devices wirelesslyor via a wired connection. For example, in an embodiment where one ormore user interfaces 104 are external to the apparatus 100, thecommunications interface 108 may communicate with the one or moreexternal user interfaces 104 wirelessly or via a wired connection.Similarly, in an embodiment where one or more memories 106 are externalto the apparatus 100, the communications interface 108 may communicatewith the one or more external memories 106 wirelessly or via a wiredconnection.

It will be appreciated that FIG. 1 only shows the components required toillustrate this aspect of the disclosure, and in a practicalimplementation the apparatus 100 may comprise additional components tothose shown. For example, the apparatus 100 may comprise a battery orother power supply for powering the apparatus 100 or means forconnecting the apparatus 100 to a mains power supply.

FIG. 2 shows a block diagram illustrating a method 200 for adjusting amodel of an anatomical structure in a sequence of images of theanatomical structure. The illustrated method 200 can generally beperformed by or under the control of the processor 102 of the apparatus100. The method 200 may, for example, be a computer-implemented method.

Briefly, the method 200 comprises placing the model of the anatomicalstructure with respect to the anatomical structure in the sequence ofimages (at block 202 of FIG. 2), receiving a user input to adjust (ormodify) the model in a selected image of the sequence (at block 204 ofFIG. 2), and adjusting (or modifying), based on the user input, a partof the model that lies in the selected image and a previously unadjusted(or unmodified) part of the model that lies in one or more other imagesof the sequence, whilst fixing (or freezing) in place a previouslyadjusted (or modified) part of the model that lies in other images ofthe sequence (at block 206 of FIG. 2).

As used herein, the terms ‘previously adjusted’ and ‘previouslyunadjusted’ refer to whether a part of the model has already beenadjusted by a user, for example, based on one or more earlier userinputs received in the manner described with reference to block 204 ofFIG. 2. More specifically, a previously adjusted part of the model is apart of the model that has previously been adjusted from a received userinput directed to that part of the model and an unadjusted part of themodel is a part of the model that has not previously been adjusted froma user input directed to that part of the model. It is noted that theterm ‘previously adjusted’ thus does not necessarily include parts ofthe model that have been automatically adjusted due to propagation of anadjustment made by the user to another part of the model in anotherimage.

In this way, changes made by the user to the model are only propagatedto parts of the model lying in images of the sequence that have notalready been adjusted by the user. Thus, parts of the model lying inimages of the sequence that have been previously adjusted are fixed (orfrozen) in place. This eliminates the need for the user to have torepeatedly check and re-adjust parts of the model that have alreadypreviously been adjusted and approved by the user. In this way, theworkflow is simpler and more efficient, saving time for the user, andalso produces more reliably adjusted models of anatomical structuressince errors propagating through images in the sequence are avoided.

The anatomical structure referred to herein may be any anatomicalstructure. For example, the anatomical structure may be an organ, suchas, for example, a heart, a lung or a pair of lungs, a brain, anintestine, a liver, a kidney, or any other anatomical structure, or anycombination of anatomical structure. The anatomical structure cancomprise one or more anatomical parts. For example, images of the heartcan comprise a ventricle, an atrium, an aorta, or any other part of theheart, or any combination of parts of the heart.

The sequence of images of the anatomical structure referred to hereinmay, for example, comprise a sequence of medical images. The images maybe acquired using any imaging modality. Examples of medical imagesinclude, but are not limited to computed tomography (CT) images (such asC-arm CT images, Spectral CT images, or Phase Contrast CT images),magnetic resonance (MR) images, X-ray images, ultrasound (US) images,fluoroscopy images, positron emission tomography (PET) images, singlephoton emission computed tomography (SPECT) images, nuclear medicineimages, or any other medical images. In general, the images in thesequence of images can be of any image format (for example, a DigitalImaging and Communications in Medicine DICOM image format, or any otherimage format). For example, the image format may be any format suitablefor allowing a model to be placed in the sequence of images.

In any of the embodiments described herein, the sequence of images can,for example, comprise a sequence of two-dimensional images, such as atime sequence of two-dimensional images. In these embodiments, eachtwo-dimensional image in the sequence may show the anatomical structureat a subsequent point in time from the preceding two-dimensional imagein the sequence. For example, in some embodiments, the sequence ofimages may comprise a sequence of two-dimensional images of a part of aheart, showing the heart over time in two-dimensions, at differentstages of a heartbeat.

Alternatively, in any of the embodiments described herein, the sequenceof images can, for example, comprise a sequence of two dimensionalslices through a three-dimensional image. In these embodiments, eachslice of the three-dimensional image in the sequence may show theanatomical structure at a subsequent two-dimensional spatial plane fromthe preceding slice of the three-dimensional image in the sequence. Insome embodiments, for example, a three-dimensional image may bedisplayed to a user as a series of two-dimensional slices through thethree-dimensional image volume. For example, where a series oftwo-dimensional slices (or scans) are taken to build up athree-dimensional volume (for example, in a computed tomography CTscan), the slices may be presented to the user in the sequence in whichthe image slices are taken. However, the sequence of two-dimensionalslices through the three-dimensional volume does not necessarily have toalign with the imaging direction. For example, in some embodiments, thetwo-dimensional slices through the three dimensional volume may be takenat any (arbitrary) angle with respect to the edges of the volume.

Alternatively, in any of the embodiments described herein, the sequenceof images can, for example, comprise a sequence of three-dimensionalimages, such as a time sequence of three-dimensional images. In theseembodiments, each three-dimensional image in the sequence may show theanatomical structure at a subsequent point in time from the precedingthree-dimensional image in the sequence. For example, in someembodiments, the sequence of images may be a sequence ofthree-dimensional images of a part of a heart, showing the heart overtime in three-dimensions, at different stages of a heartbeat.

Although examples have been provided for the type of images in thesequence and for the anatomical structure in the sequence of images, itwill be understood that the apparatus and method described herein mayalso be used for adjusting a model of any other anatomical structure inany other type of image.

Turning now to the model of the anatomical structure, in any of theembodiments described herein, the model of the anatomical structure maybe, for example, a geometric model representing a shape of an anatomicalstructure. The model may, for example, be a model used in a model-basedsegmentation of the anatomical structure in the sequence of images.Thus, in some embodiments, the model can be a segmentation model. Themodel can be a deformable model. In embodiments where the sequence ofimages comprises a sequence of two-dimensional images, the model may bea two-dimensional model. In embodiments where the sequence of imagescomprises a sequence of two dimensional slices through athree-dimensional image or a sequence of images a sequence ofthree-dimensional images, the model may be a three-dimensional model.

The model of the anatomical structure can comprise a plurality ofsegments. In some embodiments, for example, the model can comprise amesh. In some embodiments, the mesh can be a polygon mesh. In theseembodiments, the model can thus comprise a plurality of polygonsegments. For example, the mesh may be a triangular mesh according tosome embodiments. In these embodiments, the model can thus comprise aplurality of triangular segments. However, it will be understood thatany other polygon mesh can be used for the model. In some embodiments,the plurality of segments (or the mesh) may be structured. For example,the plurality of segments may each be the same shape and optionally thesame size. In other embodiments, the plurality of segments (or the mesh)may be unstructured. For example, one or more of the plurality ofsegments may be different shapes, one or more of the plurality ofsegments may be different sizes, or both. In some embodiments, the model(for example, the mesh) may comprise a plurality of control points. Inthese embodiments, each control point can define a point on a surface ofthe anatomical structure that is represented by the model.

Although examples have been provided for the type of model, a personskilled in the art will appreciate that other types of models may beused herein. For example, in some embodiments, the model may be a mask,such as a labelled mask, or any other type of model of which the personskilled in the art will be aware.

In embodiments where the sequence of images comprises a time sequence oftwo-dimensional images, each image in the sequence may show atwo-dimensional representation of the same part of the anatomicalstructure at a different point in time. In these embodiments, the modelof the anatomical structure may comprise a plurality of two-dimensionalrepresentations of the anatomical structure and there may be arepresentation for each of the two-dimensional images in the sequence.In some embodiments, the plurality of representations in the model maybe linked to each other such that adjusting a part of the model in afirst two-dimensional image of the sequence, adjusts an equivalent partof the model in another representation of the anatomical structure thatis previously unadjusted and lies in another two-dimensional image ofthe sequence.

Similarly, in embodiments where the sequence of images comprises asequence of three-dimensional images, each image in the sequence mayshow a three-dimensional representation of the same part of theanatomical structure at a different point in time. In these embodiments,the model of the anatomical structure may comprise a plurality ofthree-dimensional representations of the anatomical structure and theremay be a representation for each of the three-dimensional images in thesequence. In some embodiments, the plurality of representations in themodel may be linked to each other such that adjusting a part of themodel in a first three-dimensional image of the sequence, adjusts anequivalent part of the model in another representation of the anatomicalstructure that is previously unadjusted and lies in anotherthree-dimensional image of the sequence.

Returning back to FIG. 2, as mentioned earlier, at block 202 of FIG. 2,the model of the anatomical structure is placed (or positioned) withrespect to the anatomical structure in the sequence of images. In someembodiments, placing the model with respect to the anatomical structurein the sequence of images at block 202 may comprise detecting theanatomical structure in the sequence of images to place the model at thelocation of the anatomical structure in the sequence of images. Forexample, where the anatomical structure is a heart, this can comprisedetecting the heart, or a part of the heart (such as the mitral valve,or any other part of the heart), in the sequence of images. Theanatomical structure may be detected in the sequence of images using anysuitable feature extraction technique (such as the Generalized HoughTransformation GHT, or any other feature extraction technique).

Alternatively or in addition, in some embodiments, placing the modelwith respect to the anatomical structure in the sequence of images atblock 202 of FIG. 2 may comprise segmenting the anatomical structure inthe sequence of images. For example, the segmentation may comprisefitting the model to one or more parts of an anatomical structure,and/or one or more anatomical structures, in the sequence of images.This can be performed using any suitable segmentation technique (such asany model-based segmentation technique) and the person skilled in theart will be familiar with various segmentation methods that might beused.

Although not illustrated in FIG. 2, in some embodiments, the method 200may further comprise rendering or displaying one or more images from thesequence of images on a user interface 104 (such as a display device)for presentation to the user. The one or more images from the sequencecan be rendered or displayed with the model placed in the one or moreimages. For example, a representation of at least part the model may berendered or displayed overlaying the one or more images.

In some embodiments, the images in the sequence can be displayed to theuser in sequence (e.g. in a predetermined order) and thus the parts ofthe model that are previously adjusted or previously unadjusted dependon the ordering of the sequence, as will be described below.

At block 204 of FIG. 2, as mentioned earlier, a user input is receivedto adjust the model in a selected image of the sequence. In someembodiments, a user input to adjust the model in a selected image of thesequence may comprise receiving a user input indicating an adjustment topart of the model in the selected image. For example, the user may “dragand drop” part of the model (for example, a control point of the model)from one point in the selected image to another point in the selectedimage, using a user interface 104 (for example, a mouse, a touch screen,or any other user interface, such as any of those described earlier).The user input may provide an indication of a path or a vector throughwhich to move the part of the model in the selected image.

Generally, the selected image referred to herein is the image that iscurrently being viewed, assessed and/or in which a part of the model isbeing adjusted by a user. In some embodiments, the selected image can beany image in the sequence of images that is selected by the user. Forexample, in some embodiments, the user may select in which images in thesequence to adjust a part of the model of the anatomical structure. Insome of these embodiments, the user may also select the order of imagesin the sequence in which to adjust a part of the model of the anatomicalstructure (and, as such, the user may choose to view the images of thesequence in any order). The user may select an image via a userinterface 104. The processor 102 may receive an indication of theselection from the user interface 104.

In other embodiments, the selected image can be the current image in thesequence that is displayed to the user for the user to adjust a part ofthe model of the anatomical structure in the image. In some embodiments,each image may be displayed to the user in a predefined order for theuser to adjust a part of the model of the anatomical structure in thoseimages. For example, the predefined order may be a predefined timeorder, where the images in the sequence are displayed to the user in achronological order or in a reverse chronological order.

In some embodiments where the sequence of images comprises a sequence oftwo-dimensional slices through a three-dimensional image volume, thepredefined order may be a sequential order through the three-dimensionalimage volume, where the two-dimensional slices in the sequence aredisplayed to the user in the sequential order in which they appear inthe three-dimensional image volume. For example, the two-dimensionalslices may be displayed to the user in a predefined order that starts,for example, from one edge of the three-dimensional image volume andends, for example, at an opposite edge of the three-dimensional imagevolume.

In some examples, the sequential order of two-dimensional slices throughthe three-dimensional image volume may be a sequential order oftwo-dimensional slices that are perpendicular to an axis associated withan anatomical feature of the anatomical structure in thethree-dimensional image. For example, in some embodiments where theanatomical structure is a heart, the axis may be defined as an axis thatruns through the left ventricle from the center of the mitral valveplane to the apex in the three-dimensional image. In this example, thesequence of images may then be displayed in an order to the user, eitheran order from the mitral valve plane to the apex or, conversely, anorder from the apex to the mitral valve plane. In this way, the sequenceof images is displayed to the user in a meaningful or intuitive way. Inanother example, instead of moving sequentially through a wholethree-dimensional image volume, a long axis two-dimensional slice may bedisplayed initially and this may be followed by the display of aspecific stack of short axis two-dimensional slices.

Although examples have been provided for an order in which the images ofthe sequence may be displayed to the user, the person skilled in the artwill appreciate that other display orders are also possible.

Returning back to FIG. 2, at block 206, as mentioned earlier, a part ofthe model that lies in the selected image is adjusted based on the userinput that is received and a previously unadjusted part of the modelthat lies in one or more other images of the sequence is adjusted basedon the user input, whilst a previously adjusted part of the model thatlies in other images of the sequence is fixed in place.

In some embodiments, adjusting, based on the user input, may compriseadjusting a part of the model that lies in the selected image and thatis indicated by the user in the user input. For example, the user inputmay indicate movement of a part of the model that lies in the selectedimage to a new (or adjusted) position, where the new position is alsoindicated by the user. In this way, the position of a part of the modelthat lies in the selected image can be adjusted (for example,corrected), according to instructions provided by the user input.

In embodiments where the model of the anatomical structure comprises amesh, which itself comprises a plurality of segments, the method mayfurther comprise determining which of the plurality of segments lie in(or belong to) the selected image in the sequence and adjusting, basedon the user input, one or more of the segments that are determined tolie in the selected image in the sequence. It may be determined, forexample, that a segment of the model lies in (or belongs to) theselected image if more than a predefined percent (for example, 50percent) of that segment lies in the selected image. In someembodiments, the one or more adjusted segments lying in the selectedimage may comprise at least one segment of the mesh to which thereceived user input relates and one or more adjacent segments.

In some embodiments, adjusting, based on the user input, a previouslyunadjusted part of the model that lies in one or more other images ofthe sequence may comprise propagating an adjustment indicated by theuser input for a part of the model that lies in the selected image toone or more previously unadjusted parts of the model that lie in one ormore other images of the sequence. For example, if it can be inferredfrom the adjustment to the part of the model in the selected image thatone or more other parts of the model are to be adjusted, based on theuser input, then the one or more other parts of the model are adjustedin the images in which those parts of the model lie, provided that thoseparts of the model have not previously been adjusted by the user. Itmay, for example, be inferred from the adjustment to the part of themodel in the selected image that one or more other parts of the modelare to be adjusted where the one or more other parts of the model arelinked (for example, functionally linked and/or geometrically linked) tothe part of the model in the selected image.

In some embodiments, adjusting, based on the user input, a previouslyunadjusted part of the model that lies in one or more other images ofthe sequence may comprise propagating an adjustment indicated by theuser input for a part of the model that lies in the selected image toone or more previously unchecked parts of the model (e.g. parts of themodel that have not yet been looked at by the user) that lie in one ormore other images of the sequence. In this way, if it is identified thatno adjustments have been made by the user to a part of the model thatlies in a particular image that has been checked by the user, then thepart of the model that lies in this particular image is effectivelyfixed (or frozen) in place. Thus in these embodiments, any subsequentadjustments made, based on a user input, to a part of the model thatlies in a selected image are not propagated to previously checked partsof the model that lie in one or more other images of the sequence. Inthis way, the user can be certain that any parts of the model that havebeen previously checked (even if they have not been adjusted) are fixedin place and thus do not need to be checked again.

In some embodiments, a previously unadjusted part of the model that liesin one or more other images of the sequence may be adjusted by using aninterpolation to adjust the previously unadjusted part of the model thatlies in the one or more other images of the sequence, based on theadjustment to the part of the model that lies in the selected image. Forexample, one or more previously unadjusted parts of the model may alsobe adjusted in order to ensure one or more parts of the model have asmooth profile and/or in order to prevent discontinuities beingintroduced into the model.

The propagation of an adjustment made to a particular part of the modelthat lies in the selected image of the sequence to other previouslyunadjusted parts of the model that lie in one or more other images ofthe sequence can be achieved in a variety of ways. In some embodiments,for example, the adjustment may be propagated to one or more otherimages in which a previously unadjusted part of the model lies that arelocated within a certain radius of the selected image (for example,within a certain number of images preceding and/or subsequent to, theselected image in the sequence). For example, in some embodiments, anadjustment may only be propagated to images that are immediateneighboring images of the selected image in the sequence of images, oran adjustment may be propagated to a predetermined number x of theneighboring images in the sequence, where x is an integer value. In someembodiments, the adjustment may be propagated to images preceding and/orsubsequent to the selected image in the sequence that were taken (e.g.recorded or scanned) within a predetermined time interval of theselected image.

In some embodiments, the one or more other images of the sequence (inwhich a previously unadjusted part of the model is adjusted) can beselected based on a property of the anatomical structure. In someembodiments, the property may be a functional property of the anatomicalstructure, such as the manner in which the anatomical property changesover time. For example, in embodiments where the anatomical structure isthe heart, the property may be the heart rate. In an example embodiment,the sequence of images of an anatomical structure comprises a timesequence of two-dimensional images of a heart, where the heart beats ata rate of 60 beats per-minute. The heart in this example embodimentundergoes a full cardiac cycle each second. As such, any adjustmentsthat are made to a part of the model of the heart in a selected image inthe time-sequence of two-dimensional images are thus not propagatedindefinitely through the time sequence of two-dimensional images, as theadjustments only apply to the particular point of the cardiac cycle thatis shown in the selected image. In such embodiments, adjustments may bepropagated to other images in the sequence of images corresponding tothe same point of the cardiac cycle as the selected image.

In some embodiments, a size of an adjustment made by the user to a partof the model in the selected image may be weighted before the adjustmentis also made to a previously unadjusted part of the model that lies inone or more other images. For example, the weighting applied to theadjustment may reduce the extent of the adjustment to the part of themodel that lies in the one or more other images. In some embodiments,for example, the weighting applied to an adjustment may be based on thedistance of each of the one or more other images from the selected imagein the sequence such that the greater the distance between the selectedimage and the other image, the greater the reduction in the extent ofthe adjustment to the part of the model that lies in the other image. Insome embodiments, the weighting applied to an adjustment may beproportional to the number of images in the sequence that are locatedbetween the selected image and the other image in which the previouslyunadjusted portion of the model lies. For example, the adjustment may bereduced exponentially in relation to the distance between the selectedimage and the other image.

In an embodiment where the anatomical structure is a heart, for example,the weighting may be proportional to (or may reflect) the phase of thecardiac cycle. Thus, if the heart is expected to contract by, forexample, 30 percent between the selected image and the next image in thesequence, then the adjustment may be reduced by the same percentage.This may be used to ensure that an adjustment made to an image taken ata point in time corresponding to systole (e.g. contraction) of the heartis not applied to parts of the model in an image corresponding todiastole (e.g. relaxation) of the heart.

In some embodiments, as noted previously, a user can be permitted toview images of the sequence in any order and thus the user may selectand adjust a part of the model in any image of the sequence. Consideringan example of a sequence of two-dimensional slices through athree-dimensional image, if the user were to select two-dimensionalslice number 13 of the sequence as an image in which to adjust a part ofthe model and the parts of the model lying in neighboringtwo-dimensional slices have not previously been adjusted, then theadjustment is propagated to the neighboring slices in three-dimensionsin both directions (for example, to slice number 13−− and slice number13++). If, in the next adjustment, the user selects a part of the modellying in two-dimensional slice number 10 to adjust, the adjustment tothe part of the model lying in slice number 10 will not be propagated toslice number 13 because slice number 13 was previously adjusted by theuser in the earlier operation. The adjustment to the part of the modellying in slice number 10 will, however, be propagated to slice numbers1-10, 11, 12 and 14++ because these were previously unadjusted by theuser.

In some embodiments, the previously unadjusted part of the model may liein one or more images that are subsequent to the selected image in thesequence and the previously adjusted part of the model may lie in imagesthat precede the selected image in the sequence. For example, in someembodiments where the sequence of images are displayed to the user in apredefined order (as described earlier), adjusting at block 206 of FIG.2 can comprise adjusting, based on the user input, a part of the modelthat lies in the selected image and a part of the model that lies in oneor more images that are subsequent to the selected image according tothe predefined order, whilst fixing in place a part of the model thatlies in images that precede the selected image according to thepredefined order. Thus, in some embodiments, adjustments are propagatedin one direction through the sequence of images to images in thesequence in which the model has not yet been adjusted by the user, whileadjustments are not propagated to preceding images in the sequence inwhich a part of the model has already been adjusted by the user. In suchembodiments, therefore, the previously adjusted part of the model liesin images that precede the selected image in the sequence and thepreviously unadjusted part of the model lies in one or more images thatare subsequent to the selected image in the sequence.

In this way, the model adjustment only affects the parts of the modelthat belong to the current image in which a part of the model is beingadjusted and to those images in which a part of the model has not yetbeen adjusted (for example, those images in which a part of the modelmay still need adjustment in the future). The parts of the model thathave already been adjusted remain unchanged. As a result, the user canbe sure that current adjustments do not introduce errors in alreadyadjusted parts of the model and the user does not have to check andre-adjust parts of the model in previous images again. Thus, theworkflow of the user is made more efficient.

In any of the embodiments described herein, once the adjustments havebeen made to the selected image, in some embodiments, the method 200further comprises fixing in place the adjusted part of the model thatlies in the selected image for subsequent adjustments. The method 200may then be repeated on one or more subsequent images, for example, themethod 200 may further comprise receiving at least one further userinput to adjust the model in at least one further image in the sequenceand adjusting, based on the at least one further user input, a part ofthe model that lies in the at least one further image, whilst fixing inplace a previously adjusted part of the model that lies in other imagesof the sequence.

In this way, once the user has adjusted a part of the model in theimage, that part of the model is effectively “frozen”. Subsequentadjustments to other parts of the model in the image may be propagatedthrough to other images in the sequence, but will not affect parts ofthe model that have already been adjusted (and, for example, checked) bythe user. This makes the adjustment (and, for example, checking) processmore efficient by ensuring that the user does not have to re-adjustparts of the model that have already been adjusted.

FIG. 3 illustrates an example embodiment of a method of adjusting amodel of an anatomical structure in use according to an embodiment. Morespecifically, the example embodiment illustrated in FIG. 3 shows athree-dimensional short axis cine magnetic resonance MR image of aheart. The heart has been segmented using a three-dimensional model.Although not illustrated in FIG. 3, the three-dimensional modelcomprises a mesh, which itself comprises a plurality of segments (forexample, a plurality of triangular segments).

The three-dimensional image comprises a sequence of two-dimensionalslices. The two-dimensional slices 304 to 328 are displayed to the userin an order, starting from the top of the three-dimensional image in aplane that is perpendicular to the plane of FIG. 3, along the imagingdirection of the arrow 302. The imaging direction shown in FIG. 3 isfrom the mitral valve plane to the apex. In this example embodiment, thetwo-dimensional slices correspond to the imaging slices of the MRimaging technique and the two-dimensional slices in the sequence aredisplayed to the user in the imaging direction, such that thetwo-dimensional slice that was acquired first during imaging is thefirst two-dimensional slice to be displayed to the user and thetwo-dimensional slice that was acquired last during imaging is the lasttwo-dimensional slice to be displayed to the user.

As mentioned earlier, although not illustrated in FIG. 3, thethree-dimensional model comprises a mesh, which itself comprises aplurality of segments. The segments of the mesh overlay the anatomicalstructure in the sequence of two-dimensional slices and the segments ofthe mesh can be adjusted by the user to adjust parts of the model. Inthe example embodiment of FIG. 3, the two-dimensional slice 314 is theselected slice, which is the current slice in which a part of the modelis being adjusted by the user. The two-dimensional slices 304, 306, 308,310, and 312 are slices in which a part of the model has previously beenadjusted by the user and are therefore fixed in place. In someembodiments, the slices 304, 306, 308, 310, and 312 may be marked as“frozen”.

The two-dimensional slices 318 to 328 are slices in which a part of themodel is previously unadjusted by the user. In practice, in the exampleembodiment of FIG. 3, for each adjustment that is made by the user tothe part of the model (or, more specifically, to the mesh of the model)that lies in the selected slice 314, an adjustment is also made to apart of the model lying in one or more previously unadjusted parts ofthe model (or, more specifically, one or more parts of the mesh of themodel) lying in one or more of the slices 318 to 328. The user mayadjust the part of the model that lies in the selected slice 314 byadjusting one or more control points of the model. Once the user hasfinished adjusting the part of the model that lies in the selected slice314, the part of the model belonging to slice 314 is classed as apreviously adjusted part of the model, which is then fixed in place toprevent future adjustments to that part of the model. Then, in thisexample embodiment, the next slice 316 in the sequence oftwo-dimensional slices is displayed for adjustment by the user in thesame way as described earlier for slice 314. This process can berepeated for each image slice 318 to 328 in which a previouslyunadjusted part of the model lies in order. In this way, the model ofthe anatomical structure is adjusted in the sequence of two-dimensionalslices according to the method described herein.

In any of the embodiments described herein, the previously adjustedparts of the model that are fixed in place may be color-coded in adifferent color to the previously unadjusted parts of the model. In someembodiments, the part of the model that is being adjusted in theselected image may be color-coded in a different color to the previouslyunadjusted parts of the model and/or the previously adjusted parts ofthe model. In embodiments where the model comprises a mesh, which itselfcomprises a plurality of segments, the segments of the mesh thatcorrespond to a part of the model that is fixed in place may becolor-coded in a different color to segments of the mesh that correspondto a part of the model that is previously unadjusted to indicate to theuser that those parts of the model are fixed in place. If the model isdisplayed at a user interface 104 (for example, in a viewer of a userinterface 104), the user is provided with immediate knowledge of whichparts of the model can still be adjusted.

There is therefore provided an improved method and apparatus foradjusting a model of an anatomical structure in a sequence of images ofthe anatomical structure.

There is also provided a computer program product comprising a computerreadable medium, the computer readable medium having computer readablecode embodied therein, the computer readable code being configured suchthat, on execution by a suitable computer or processor, the computer orprocessor is caused to perform the method or methods described herein.Thus, it will be appreciated that the disclosure also applies tocomputer programs, particularly computer programs on or in a carrier,adapted to put embodiments into practice. The program may be in the formof a source code, an object code, a code intermediate source and anobject code such as in a partially compiled form, or in any other formsuitable for use in the implementation of the method according to theembodiments described herein.

It will also be appreciated that such a program may have many differentarchitectural designs. For example, a program code implementing thefunctionality of the method or system may be sub-divided into one ormore sub-routines. Many different ways of distributing the functionalityamong these sub-routines will be apparent to the skilled person. Thesub-routines may be stored together in one executable file to form aself-contained program. Such an executable file may comprisecomputer-executable instructions, for example, processor instructionsand/or interpreter instructions (e.g. Java interpreter instructions).Alternatively, one or more or all of the sub-routines may be stored inat least one external library file and linked with a main program eitherstatically or dynamically, e.g. at run-time. The main program containsat least one call to at least one of the sub-routines. The sub-routinesmay also comprise function calls to each other.

An embodiment relating to a computer program product comprisescomputer-executable instructions corresponding to each processing stageof at least one of the methods set forth herein. These instructions maybe sub-divided into sub-routines and/or stored in one or more files thatmay be linked statically or dynamically. Another embodiment relating toa computer program product comprises computer-executable instructionscorresponding to each means of at least one of the systems and/orproducts set forth herein. These instructions may be sub-divided intosub-routines and/or stored in one or more files that may be linkedstatically or dynamically.

The carrier of a computer program may be any entity or device capable ofcarrying the program. For example, the carrier may include a datastorage, such as a ROM, for example, a CD ROM or a semiconductor ROM, ora magnetic recording medium, for example, a hard disk. Furthermore, thecarrier may be a transmissible carrier such as an electric or opticalsignal, which may be conveyed via electric or optical cable or by radioor other means. When the program is embodied in such a signal, thecarrier may be constituted by such a cable or other device or means.Alternatively, the carrier may be an integrated circuit in which theprogram is embedded, the integrated circuit being adapted to perform, orused in the performance of, the relevant method.

Variations to the disclosed embodiments can be understood and effectedby those skilled in the art, from a study of the drawings, thedisclosure and the appended claims. In the claims, the word “comprising”does not exclude other elements or steps, and the indefinite article “a”or “an” does not exclude a plurality. A single processor or other unitmay fulfil the functions of several items recited in the claims. Themere fact that certain measures are recited in mutually differentdependent claims does not indicate that a combination of these measurescannot be used to advantage. A computer program may bestored/distributed on a suitable medium, such as an optical storagemedium or a solid-state medium supplied together with or as part ofother hardware, but may also be distributed in other forms, such as viathe Internet or other wired or wireless telecommunication systems. Anyreference signs in the claims should not be construed as limiting thescope.

The invention claimed is:
 1. A method for adjusting a model of ananatomical structure in a sequence of images of the anatomicalstructure, the method comprising: placing the model with respect to theanatomical structure in the sequence of images; receiving a user inputto adjust the model in a selected image of the sequence; and adjusting,based on the user input, a part of the model that lies in the selectedimage and a previously unadjusted part of the model that lies in one ormore other images of the sequence, whilst fixing in place a previouslyadjusted part of the model that lies in other images of the sequence. 2.A method as claimed in claim 1, wherein the previously unadjusted partof the model lies in one or more images that are subsequent to theselected image in the sequence and the previously adjusted part of themodel lies in images that precede the selected image in the sequence. 3.A method as claimed in claim 1, the method further comprising: fixing inplace the adjusted part of the model that lies in the selected image forsubsequent adjustments.
 4. A method as claimed in claim 1, whereinadjusting a previously unadjusted part of the model that lies in the oneor more other images of the sequence comprises: using an interpolationto adjust the previously unadjusted part of the model that lies in theone or more other images of the sequence, based on the adjustment to thepart of the model that lies in the selected image.
 5. A method asclaimed in claim 1, the method further comprising: receiving at leastone further user input to adjust the model in at least one further imagein the sequence; and adjusting, based on the at least one further userinput, a part of the model that lies in the at least one further image,whilst fixing in place a previously adjusted part of the model that liesin other images of the sequence.
 6. A method as claimed in claim 1,wherein the one or more other images of the sequence in which apreviously unadjusted part of the model is adjusted are selected basedon a property of the anatomical structure.
 7. A method as claimed inclaim 1, wherein the one or more other images of the sequence in which apreviously unadjusted part of the model is adjusted lie within apredetermined radius of the selected image.
 8. A method as claimed inclaim 1, wherein the sequence of images comprises: a time sequence oftwo-dimensional images, wherein each two-dimensional image in thesequence shows the anatomical structure at a subsequent point in timefrom the preceding two-dimensional image in the sequence.
 9. A method asclaimed in claim 1, wherein the sequence of images comprises: a sequenceof two-dimensional slices through a three-dimensional image, whereineach slice of the three-dimensional image in the sequence shows theanatomical structure at a subsequent two-dimensional spatial plane fromthe preceding slice of the three-dimensional image in the sequence. 10.A method as claimed in claim 1, wherein the sequence of imagescomprises: a time sequence of three-dimensional images, wherein eachthree-dimensional image in the sequence shows the anatomical structureat a subsequent point in time from the preceding three-dimensional imagein the sequence.
 11. A method as claimed in claim 1, wherein the modelof the anatomical structure comprises a mesh comprising a plurality ofsegments and wherein the method further comprises: determining which ofthe plurality of segments lie in the selected image in the sequence; andadjusting, based on the user input, one or more of the segments that aredetermined to lie in the selected image in the sequence.
 12. A method asclaimed in claim 10, wherein the one or more adjusted segments lying inthe selected image comprise at least one segment of the mesh to whichthe received user input relates and one or more adjacent segments.
 13. Acomputer program product comprising a non-transitory computer readablemedium, the non-transitory computer readable medium having computerreadable code embodied therein, the computer readable code beingconfigured such that, on execution by a suitable computer or processor,the computer or processor is caused to perform the method of claim 1.14. An apparatus for adjusting a model of an anatomical structure in asequence of images of the anatomical structure, the apparatuscomprising: a processor configured to: place the model with respect tothe anatomical structure in the sequence of images; receive a user inputto adjust the model in a selected image of the sequence; and adjust,based on the user input, a part of the model that lies in the selectedimage and a previously unadjusted part of the model that lies in one ormore other images of the sequence, whilst fixing in place a previouslyadjusted part of the model that lies in other images of the sequence.15. An apparatus as claimed in claim 14, wherein the processor isconfigured to: control a user interface to render the adjusted model ofthe anatomical structure; and/or control a memory to store the adjustedmodel of the anatomical structure.